Mutual Information-Based Modified Randomized Weights Neural Networks

Randomized weights neural networks have fast learning speed and good generalization performances with one single hidden layer structure. Input weighs of the hidden layer are produced randomly. By employing certain activation functions, outputs of the hidden layers are calculated with some randomization. Output weights are computed using pseudoinverse. Mutual information can be used to measure mutual dependence of the two variables quantitatively based on the probability theory. In this paper, these hidden layers’ outputs that relate to prediction variable closing are selected with the simple mutual information based feature selection method. These hidden nodes with high mutual information values are maintained as new hidden layer. Thus, the size of the hidden layer is reduced. The new hidden layer’s output weights are learned with the pseudoinverse method. The proposed method is compared with the original randomized algorithms using concrete compressive strength benchmark dataset.